1. Field of the Invention
This invention generally relates to the field of image processing systems and methods, and more particularly relates to methods of matching image features across multiple image views.
2. Description of Related Art
There are many image processing applications where multiple images of scenes are matched to identify common image information across the different images. For example, one way of creating three-dimensional (3-D) digital content is by analyzing multiple images of a scene. The main issue here is image matchingxe2x80x94how to automatically find the corresponding relationship among the projections in different images of the same points.
For example, as shown in FIG. 1, a left eye 102 is seen in three images 104, 106, 108, of the same scene 110. The left eye 102 appears as three respective image features 112, 114, 116, in the three images 104, 106, and 108, as shown. Image matching methods attempt to establish links 118, 120 among the three respective image features 112, 114, 116, that are common across the three images 104, 106, 108, of the same scene 110. The significance of image matching is that once a correspondence information is established, it is possible to recover the 3-D coordinates of the matched points from which more complete geometric structures can be reconstructed.
Two prior art methods have been reported which work on image pairs or triplets, respectively. For example, see the publication by R. Deriche, Z. Zhang, Q.-T. Luong and O. Faugeras, xe2x80x9cRobust Recovery of the Epipolar Geometry for an Uncalibrated Stereo Rig,xe2x80x9d Proc. European Conference on Computer Vision ""94, pp. 567-576. Additionally, see the publication by P. H. S. Torr and A. Zisserman, xe2x80x9cRobust Parameterization and Computation of the Trifocal Tensor,xe2x80x9d Image and Vision Computing, Vol. 15, No. 8, August 1997, pp. 591-605.
The basic approach is, first, to generate a number of candidate correspondences based on proximity and similarity; and then, to select the correct ones from all candidates by making sure that they satisfy an algebraic constraint (epipolar geometry in the two-view case, and trifocal tensor in the three-view case). In the terminology of estimation theory, the correct candidates are called inliers, whilst the wrong ones are called outliers. The robustness of a method is its ability to detect outliers. Unfortunately, the robustness of the two prior art methods mentioned above is limited because the constraints they are enforcing are ambiguous sometimes. That is, there may be multiple pairs or triplets of correspondences that satisfy the same instance of a constraint. Additionally, those constraints have singular conditions, e.g. when the camera positions are linear or planar. Under such cases, these two methods simply fail to work.
Therefore a need exists to overcome the problems with the prior art as discussed above, and particularly for a method and apparatus that can more successfully match features across multiple images.
According to a preferred embodiment of the present invention, an image processing system comprises a memory; a controller/processor electrically coupled to the memory; an image feature detector, electrically coupled to the controller/processor and to the memory, for detecting a plurality of image features in a first image corresponding to a first view of a scene, and for detecting a plurality of image features in at least a second image corresponding to a respective at least a second view of the scene, wherein the at least a second image deviates from the first image as a result of camera relative motion; and an image matching module, electrically coupled to the controller/processor and to the memory, for determining a two-view correspondence resulting in a potential match set of candidate image features between the first image and the at least a second image, wherein the potential match set is determined to have a maximum average strength of correspondence based at least in part on the total number of matching neighbor candidate image features for each match of the potential match set.
According to a preferred embodiment of the present invention, an image processing system comprises a memory; a controller/processor electrically coupled to the memory; an image feature detector, electrically coupled to the controller/processor and to the memory, for detecting a plurality of image features in a first image corresponding to a first view of a scene, and for detecting a plurality of image features in at least a second image corresponding to a respective at least a second view of the scene, wherein the at least a second image deviates from the first image as a result of camera relative motion; and an image matching module, electrically coupled to the controller/processor and to the memory, for determining a multiple-view correspondence between the plurality of detected features in the first image and the plurality of detected image features in the at least a second image, resulting in a potential match set of candidate image features between the first image and the at least a second image, wherein the potential match set is based at least in part on a computation of reprojection error for matched points that resulted from a projective reconstruction of the potential match set.